From: https://osf.io/p5xsd/files/osfstorage

knitr::opts_chunk$set(echo = F)

0.1 Load data:

The permutation takes to long to allow for re-computing the code each time, so load it:

1. Objective measures

1.1. Consistency:

The mean area is calculated by adding together the area for each stimulus and dividing by 29. This unit is transformed into a percentage area taking into account the different pixel resolution of each participant.
Mean area = $(Summed area / 29) * 100 / Screen area $ Where: \(Screen area = Xpixels * Ypixels\)

Note: 29 since 29 stimuli

1.2 Permuted consistency

Replicate Rothen methods. Might take some time to compute.

Calculating chance levels of consistency To create permuted datasets for each participant: the 87 xy coordinates are randomly shuffled so they are no longer linked to the original data labels (“Monday”, “5”, “April”, etc.). The mean area of the triangles based on the shuffled coordinates is computed (as described above), and the whole process is repeated 1000 times to obtain a subject-specific distribution of chance levels of consistency. A z-score is calculated comparing the observed consistency against the mean and SD of the permuted data: \(Z = [(observed consistency) – (mean consistency of permuted data)] / (SD of permuted data)\)

Code retrieved from OSF (adapted here):

1.3. SD as in Ward

As in Ward:

Specifically, the standard deviation of the x-coordinates and/or the standard deviation of the y-coordinates (measured across all trials) should exceed a proposed value of 0.075 for a normalized screen with width and height of 1 unit.

A participant who produced a horizontal straight-line form would have a very low standard deviation in the y-coordinates but a high standard deviation in x-coordinates, and a participant with a vertical line would have the reverse profile. A participant with a circular spatial form would be high on both. A participant who clicks randomly around the screen would also be high on both x and y standard deviation, but would fail the consistency tests (the triangles would be large).

2. Line intersection

An idea I have is to look into the lines and order of the forms. I would exclude when lines crosses. (since we expect forms the lines crossing means no form is formed). Needs refinement.

I think that the number of stimuli per condition should be taken into account (i.e. 9 numbers, 7 days, 12 months). Hence would need to be divided by this number of stimulus.

In each condition the connected x and y generates a segment, hence the number of segment is length(stimuli)-1. Moreover, currently, each stimuli is connected by 3 segment, one for each (of the 3) repetition. So dividing by 3, we have the average number of segment corssings per condition. Next we sum these for each ID Ideally we should compute the number of crossings across the repetitions, in addition to make it more complex it would also be computationally more demanding, and I don’t beleive it would lead to a significant difference.

To do: - Maybe the easier would be to have the average number of crossing per segment.

IMPORTANT: data frame needs to be informed of stimulus order to make sense!

3. ROC to set thresholds

We consider the questionnaires as ground truth. Since the questionnaires are scalable we still need to determine a different thresholds for the questionnaire. Also some questions might be more relevant than others to “diagnose” synesthesia (see Syn_NR).

3.1. ROC Segment intersections

3.1.1. AUC

We computed the ROC between the questionnaire with NR selected questions and the per participants averaged segment interceptions.

4.1.2. ROC and threshold

Form the ROC we used Youden’s J statistic o calculate the best threshold. This statistic maximizes the difference between true positive rate (i.e. sensitivity) and false positive rate (i.e. specificity - 1). This led to a threshold of 0.09 with a sensitivity of 84.15 and a specificity of 56.88. See Figure X:

3.1.3. Cumulative density plot

Visualize it with cumulative empirical density plots. In this plots the x xis indicates the average number of intersections per segments. The y axis indicates the percentage of participant. Hence while 93 % of the self identified synesthetes have less than 0.0927958` intersection per segment, 56 % of not self-identified synesthetes do.

3.2. ROC Consistency

3.2.1. AUC

3.2.2. ROC and threshold

Form the ROC we used Youden’s J statistic o calculate the best threshold. This statistic maximizes the difference between true positive rate (i.e. sensitivity) and false positive rate (i.e. specificity - 1). This led to a threshold of 0.16 with a sensitivity of 80.49 and a specificity of 40.26. See Figure X:

3.2.3. Cumulative density plot

Visualize it with cumulative empirical density plots. In this plots the x xis indicates the average number of intersections per segments. The y axis indicates the percentage of participant. Hence while 93 % of the self identified synesthetes have less than 1 intersection per segment, 52 % of not self-identified synesthetes do.

3.3. Power

## 
##      Two ROC curves power calculation 
## 
##          ncases = 7134
##       ncontrols = 33495
##            auc1 = 0.7179126
##            auc2 = 0.5265125
##       sig.level = 0.05
##           power = 1
##     alternative = two.sided

4. Add oder Criteris

4.1. Questinnnaire Ward

The same but with the Questionnaire threshold as defined in Ward. This led to a threshold of 0.25 with a sensitivity of 0.85 and a specificity of 0.62.

4.2. Permuted z-score consistency

Replicate Rothen methods. Might take some time to compute.

Calculating chance levels of consistency To create permuted datasets for each participant: the 87 xy coordinates are randomly shuffled so they are no longer linked to the original data labels (“Monday”, “5”, “April”, etc.). The mean area of the triangles based on the shuffled coordinates is computed (as described above), and the whole process is repeated 1000 times to obtain a subject-specific distribution of chance levels of consistency. A z-score is calculated comparing the observed consistency against the mean and SD of the permuted data: \(Z = [(observed consistency) – (mean consistency of permuted data)] / (SD of permuted data)\)

Code retrieved from OSF (adapted here):

see ´SynPermzs´

Note this might vary due to the randomisation process involved in the permutation. Some sd are 0, leadiong to NaN in the z score. The two additional lines are to take this into account.

4.3. SD threshold

With this criteria alone 93.3618844 % would qualify as 3S

5. Sankey plot for each criteria:

6. Bonus ROC for each questionnaire Criteria

## Setting levels: control = FALSE, case = TRUE
## Setting direction: controls > cases

## 
## Call:
## roc.formula(formula = SynQuest_2 ~ GA_lineInter, data = ds, percent = TRUE,     ci = TRUE, boot.n = 100, ci.alpha = 0.9, stratified = FALSE,     plot = TRUE, auc.polygon = TRUE, max.auc.polygon = TRUE,     grid = TRUE, print.auc = TRUE, show.thres = TRUE)
## 
## Data: GA_lineInter in 32364 controls (SynQuest_2 FALSE) > 8265 cases (SynQuest_2 TRUE).
## Area under the curve: 77.94%
## 95% CI: 77.48%-78.39% (DeLong)
##    threshold sensitivity specificity
## 1 0.09279577    89.47368    59.67742
## Setting levels: control = FALSE, case = TRUE
## Setting direction: controls > cases

## 
## Call:
## roc.formula(formula = SynQuest_2 ~ GA_lineInter, data = ds, percent = TRUE,     ci = TRUE, boot.n = 100, ci.alpha = 0.9, stratified = FALSE,     plot = TRUE, auc.polygon = TRUE, max.auc.polygon = TRUE,     grid = TRUE, print.auc = TRUE, show.thres = TRUE)
## 
## Data: GA_lineInter in 30711 controls (SynQuest_2 FALSE) > 9918 cases (SynQuest_2 TRUE).
## Area under the curve: 78.23%
## 95% CI: 77.78%-78.67% (DeLong)
##    threshold sensitivity specificity
## 1 0.09311506    88.59649    61.75637
## Setting levels: control = FALSE, case = TRUE
## Setting direction: controls > cases

## 
## Call:
## roc.formula(formula = SynQuest_2 ~ GA_lineInter, data = ds, percent = TRUE,     ci = TRUE, boot.n = 100, ci.alpha = 0.9, stratified = FALSE,     plot = TRUE, auc.polygon = TRUE, max.auc.polygon = TRUE,     grid = TRUE, print.auc = TRUE, show.thres = TRUE)
## 
## Data: GA_lineInter in 28797 controls (SynQuest_2 FALSE) > 11832 cases (SynQuest_2 TRUE).
## Area under the curve: 75.75%
## 95% CI: 75.28%-76.21% (DeLong)
##   threshold sensitivity specificity
## 1 0.1404563    88.97059    57.40181
## Setting levels: control = FALSE, case = TRUE
## Setting direction: controls > cases

## 
## Call:
## roc.formula(formula = SynQuest_2 ~ GA_lineInter, data = ds, percent = TRUE,     ci = TRUE, boot.n = 100, ci.alpha = 0.9, stratified = FALSE,     plot = TRUE, auc.polygon = TRUE, max.auc.polygon = TRUE,     grid = TRUE, print.auc = TRUE, show.thres = TRUE)
## 
## Data: GA_lineInter in 27231 controls (SynQuest_2 FALSE) > 13398 cases (SynQuest_2 TRUE).
## Area under the curve: 76.68%
## 95% CI: 76.23%-77.14% (DeLong)
##   threshold sensitivity specificity
## 1 0.1404563    88.31169    59.74441
## Setting levels: control = FALSE, case = TRUE
## Setting direction: controls > cases

## 
## Call:
## roc.formula(formula = SynQuest_2 ~ GA_lineInter, data = ds, percent = TRUE,     ci = TRUE, boot.n = 100, ci.alpha = 0.9, stratified = FALSE,     plot = TRUE, auc.polygon = TRUE, max.auc.polygon = TRUE,     grid = TRUE, print.auc = TRUE, show.thres = TRUE)
## 
## Data: GA_lineInter in 25752 controls (SynQuest_2 FALSE) > 14877 cases (SynQuest_2 TRUE).
## Area under the curve: 77.96%
## 95% CI: 77.52%-78.4% (DeLong)
##   threshold sensitivity specificity
## 1 0.1404563     87.7193    62.16216
## Setting levels: control = FALSE, case = TRUE
## Setting direction: controls > cases

## 
## Call:
## roc.formula(formula = SynQuest_2 ~ GA_lineInter, data = ds, percent = TRUE,     ci = TRUE, boot.n = 100, ci.alpha = 0.9, stratified = FALSE,     plot = TRUE, auc.polygon = TRUE, max.auc.polygon = TRUE,     grid = TRUE, print.auc = TRUE, show.thres = TRUE)
## 
## Data: GA_lineInter in 24186 controls (SynQuest_2 FALSE) > 16443 cases (SynQuest_2 TRUE).
## Area under the curve: 77.38%
## 95% CI: 76.93%-77.83% (DeLong)
##   threshold sensitivity specificity
## 1 0.1404563    85.18519    63.66906
## Setting levels: control = FALSE, case = TRUE
## Setting direction: controls > cases

## 
## Call:
## roc.formula(formula = SynQuest_2 ~ GA_lineInter, data = ds, percent = TRUE,     ci = TRUE, boot.n = 100, ci.alpha = 0.9, stratified = FALSE,     plot = TRUE, auc.polygon = TRUE, max.auc.polygon = TRUE,     grid = TRUE, print.auc = TRUE, show.thres = TRUE)
## 
## Data: GA_lineInter in 23403 controls (SynQuest_2 FALSE) > 17226 cases (SynQuest_2 TRUE).
## Area under the curve: 77.19%
## 95% CI: 76.74%-77.64% (DeLong)
##   threshold sensitivity specificity
## 1  0.151341    85.35354    62.82528
## Setting levels: control = FALSE, case = TRUE
## Setting direction: controls > cases

## 
## Call:
## roc.formula(formula = SynQuest_2 ~ GA_lineInter, data = ds, percent = TRUE,     ci = TRUE, boot.n = 100, ci.alpha = 0.9, stratified = FALSE,     plot = TRUE, auc.polygon = TRUE, max.auc.polygon = TRUE,     grid = TRUE, print.auc = TRUE, show.thres = TRUE)
## 
## Data: GA_lineInter in 22185 controls (SynQuest_2 FALSE) > 18444 cases (SynQuest_2 TRUE).
## Area under the curve: 76.64%
## 95% CI: 76.18%-77.1% (DeLong)
##   threshold sensitivity specificity
## 1 0.1404563    82.07547     65.4902
## Setting levels: control = FALSE, case = TRUE
## Setting direction: controls > cases

## 
## Call:
## roc.formula(formula = SynQuest_2 ~ GA_lineInter, data = ds, percent = TRUE,     ci = TRUE, boot.n = 100, ci.alpha = 0.9, stratified = FALSE,     plot = TRUE, auc.polygon = TRUE, max.auc.polygon = TRUE,     grid = TRUE, print.auc = TRUE, show.thres = TRUE)
## 
## Data: GA_lineInter in 20706 controls (SynQuest_2 FALSE) > 19923 cases (SynQuest_2 TRUE).
## Area under the curve: 76.5%
## 95% CI: 76.03%-76.97% (DeLong)
##   threshold sensitivity specificity
## 1 0.1404563    80.34934    67.22689
## Setting levels: control = FALSE, case = TRUE
## Setting direction: controls > cases

## 
## Call:
## roc.formula(formula = SynQuest_2 ~ GA_lineInter, data = ds, percent = TRUE,     ci = TRUE, boot.n = 100, ci.alpha = 0.9, stratified = FALSE,     plot = TRUE, auc.polygon = TRUE, max.auc.polygon = TRUE,     grid = TRUE, print.auc = TRUE, show.thres = TRUE)
## 
## Data: GA_lineInter in 19314 controls (SynQuest_2 FALSE) > 21315 cases (SynQuest_2 TRUE).
## Area under the curve: 76.6%
## 95% CI: 76.13%-77.07% (DeLong)
##   threshold sensitivity specificity
## 1 0.2460815    84.89796    62.16216
## Setting levels: control = FALSE, case = TRUE
## Setting direction: controls > cases

## 
## Call:
## roc.formula(formula = SynQuest_2 ~ GA_lineInter, data = ds, percent = TRUE,     ci = TRUE, boot.n = 100, ci.alpha = 0.9, stratified = FALSE,     plot = TRUE, auc.polygon = TRUE, max.auc.polygon = TRUE,     grid = TRUE, print.auc = TRUE, show.thres = TRUE)
## 
## Data: GA_lineInter in 18183 controls (SynQuest_2 FALSE) > 22446 cases (SynQuest_2 TRUE).
## Area under the curve: 76.81%
## 95% CI: 76.34%-77.29% (DeLong)
##   threshold sensitivity specificity
## 1 0.2460815    84.10853    64.11483
## Setting levels: control = FALSE, case = TRUE
## Setting direction: controls > cases

## 
## Call:
## roc.formula(formula = SynQuest_2 ~ GA_lineInter, data = ds, percent = TRUE,     ci = TRUE, boot.n = 100, ci.alpha = 0.9, stratified = FALSE,     plot = TRUE, auc.polygon = TRUE, max.auc.polygon = TRUE,     grid = TRUE, print.auc = TRUE, show.thres = TRUE)
## 
## Data: GA_lineInter in 17139 controls (SynQuest_2 FALSE) > 23490 cases (SynQuest_2 TRUE).
## Area under the curve: 77.16%
## 95% CI: 76.67%-77.64% (DeLong)
##   threshold sensitivity specificity
## 1 0.2460815    83.33333    65.98985
## Setting levels: control = FALSE, case = TRUE
## Setting direction: controls > cases

## 
## Call:
## roc.formula(formula = SynQuest_2 ~ GA_lineInter, data = ds, percent = TRUE,     ci = TRUE, boot.n = 100, ci.alpha = 0.9, stratified = FALSE,     plot = TRUE, auc.polygon = TRUE, max.auc.polygon = TRUE,     grid = TRUE, print.auc = TRUE, show.thres = TRUE)
## 
## Data: GA_lineInter in 15921 controls (SynQuest_2 FALSE) > 24708 cases (SynQuest_2 TRUE).
## Area under the curve: 76.86%
## 95% CI: 76.36%-77.36% (DeLong)
##   threshold sensitivity specificity
## 1 0.2460815    81.69014    67.21311
## Setting levels: control = FALSE, case = TRUE
## Setting direction: controls > cases

## 
## Call:
## roc.formula(formula = SynQuest_2 ~ GA_lineInter, data = ds, percent = TRUE,     ci = TRUE, boot.n = 100, ci.alpha = 0.9, stratified = FALSE,     plot = TRUE, auc.polygon = TRUE, max.auc.polygon = TRUE,     grid = TRUE, print.auc = TRUE, show.thres = TRUE)
## 
## Data: GA_lineInter in 14790 controls (SynQuest_2 FALSE) > 25839 cases (SynQuest_2 TRUE).
## Area under the curve: 77.44%
## 95% CI: 76.94%-77.94% (DeLong)
##   threshold sensitivity specificity
## 1 0.2460815    80.47138    68.82353
## Setting levels: control = FALSE, case = TRUE
## Setting direction: controls > cases

## 
## Call:
## roc.formula(formula = SynQuest_2 ~ GA_lineInter, data = ds, percent = TRUE,     ci = TRUE, boot.n = 100, ci.alpha = 0.9, stratified = FALSE,     plot = TRUE, auc.polygon = TRUE, max.auc.polygon = TRUE,     grid = TRUE, print.auc = TRUE, show.thres = TRUE)
## 
## Data: GA_lineInter in 14268 controls (SynQuest_2 FALSE) > 26361 cases (SynQuest_2 TRUE).
## Area under the curve: 76.86%
## 95% CI: 76.35%-77.37% (DeLong)
##   threshold sensitivity specificity
## 1  0.332956    82.17822    65.85366
## Setting levels: control = FALSE, case = TRUE
## Setting direction: controls > cases

## 
## Call:
## roc.formula(formula = SynQuest_2 ~ GA_lineInter, data = ds, percent = TRUE,     ci = TRUE, boot.n = 100, ci.alpha = 0.9, stratified = FALSE,     plot = TRUE, auc.polygon = TRUE, max.auc.polygon = TRUE,     grid = TRUE, print.auc = TRUE, show.thres = TRUE)
## 
## Data: GA_lineInter in 13224 controls (SynQuest_2 FALSE) > 27405 cases (SynQuest_2 TRUE).
## Area under the curve: 75.54%
## 95% CI: 75.01%-76.06% (DeLong)
##   threshold sensitivity specificity
## 1  0.332956          80    65.13158
## Setting levels: control = FALSE, case = TRUE
## Setting direction: controls > cases

## 
## Call:
## roc.formula(formula = SynQuest_2 ~ GA_lineInter, data = ds, percent = TRUE,     ci = TRUE, boot.n = 100, ci.alpha = 0.9, stratified = FALSE,     plot = TRUE, auc.polygon = TRUE, max.auc.polygon = TRUE,     grid = TRUE, print.auc = TRUE, show.thres = TRUE)
## 
## Data: GA_lineInter in 12006 controls (SynQuest_2 FALSE) > 28623 cases (SynQuest_2 TRUE).
## Area under the curve: 75.59%
## 95% CI: 75.05%-76.13% (DeLong)
##   threshold sensitivity specificity
## 1 0.2460815    75.68389    68.84058
## Setting levels: control = FALSE, case = TRUE
## Setting direction: controls > cases

## 
## Call:
## roc.formula(formula = SynQuest_2 ~ GA_lineInter, data = ds, percent = TRUE,     ci = TRUE, boot.n = 100, ci.alpha = 0.9, stratified = FALSE,     plot = TRUE, auc.polygon = TRUE, max.auc.polygon = TRUE,     grid = TRUE, print.auc = TRUE, show.thres = TRUE)
## 
## Data: GA_lineInter in 11049 controls (SynQuest_2 FALSE) > 29580 cases (SynQuest_2 TRUE).
## Area under the curve: 75.9%
## 95% CI: 75.35%-76.45% (DeLong)
##   threshold sensitivity specificity
## 1  0.332956    77.35294    66.92913
## Setting levels: control = FALSE, case = TRUE
## Setting direction: controls > cases

## 
## Call:
## roc.formula(formula = SynQuest_2 ~ GA_lineInter, data = ds, percent = TRUE,     ci = TRUE, boot.n = 100, ci.alpha = 0.9, stratified = FALSE,     plot = TRUE, auc.polygon = TRUE, max.auc.polygon = TRUE,     grid = TRUE, print.auc = TRUE, show.thres = TRUE)
## 
## Data: GA_lineInter in 9657 controls (SynQuest_2 FALSE) > 30972 cases (SynQuest_2 TRUE).
## Area under the curve: 78.5%
## 95% CI: 77.98%-79.02% (DeLong)
##   threshold sensitivity specificity
## 1  0.332956    76.40449    70.27027
## Setting levels: control = FALSE, case = TRUE
## Setting direction: controls > cases

## 
## Call:
## roc.formula(formula = SynQuest_2 ~ GA_lineInter, data = ds, percent = TRUE,     ci = TRUE, boot.n = 100, ci.alpha = 0.9, stratified = FALSE,     plot = TRUE, auc.polygon = TRUE, max.auc.polygon = TRUE,     grid = TRUE, print.auc = TRUE, show.thres = TRUE)
## 
## Data: GA_lineInter in 9048 controls (SynQuest_2 FALSE) > 31581 cases (SynQuest_2 TRUE).
## Area under the curve: 79.25%
## 95% CI: 78.74%-79.76% (DeLong)
##   threshold sensitivity specificity
## 1 0.2460815    73.27824          75
## Setting levels: control = FALSE, case = TRUE
## Setting direction: controls > cases

## 
## Call:
## roc.formula(formula = SynQuest_2 ~ GA_lineInter, data = ds, percent = TRUE,     ci = TRUE, boot.n = 100, ci.alpha = 0.9, stratified = FALSE,     plot = TRUE, auc.polygon = TRUE, max.auc.polygon = TRUE,     grid = TRUE, print.auc = TRUE, show.thres = TRUE)
## 
## Data: GA_lineInter in 8439 controls (SynQuest_2 FALSE) > 32190 cases (SynQuest_2 TRUE).
## Area under the curve: 78.74%
## 95% CI: 78.21%-79.26% (DeLong)
##   threshold sensitivity specificity
## 1 0.2460815    72.43243    75.25773
## Setting levels: control = FALSE, case = TRUE
## Setting direction: controls > cases

## 
## Call:
## roc.formula(formula = SynQuest_2 ~ GA_lineInter, data = ds, percent = TRUE,     ci = TRUE, boot.n = 100, ci.alpha = 0.9, stratified = FALSE,     plot = TRUE, auc.polygon = TRUE, max.auc.polygon = TRUE,     grid = TRUE, print.auc = TRUE, show.thres = TRUE)
## 
## Data: GA_lineInter in 7917 controls (SynQuest_2 FALSE) > 32712 cases (SynQuest_2 TRUE).
## Area under the curve: 78.64%
## 95% CI: 78.1%-79.17% (DeLong)
##   threshold sensitivity specificity
## 1 0.2460815    71.80851    75.82418
## Setting levels: control = FALSE, case = TRUE
## Setting direction: controls > cases

## 
## Call:
## roc.formula(formula = SynQuest_2 ~ GA_lineInter, data = ds, percent = TRUE,     ci = TRUE, boot.n = 100, ci.alpha = 0.9, stratified = FALSE,     plot = TRUE, auc.polygon = TRUE, max.auc.polygon = TRUE,     grid = TRUE, print.auc = TRUE, show.thres = TRUE)
## 
## Data: GA_lineInter in 7308 controls (SynQuest_2 FALSE) > 33321 cases (SynQuest_2 TRUE).
## Area under the curve: 78.78%
## 95% CI: 78.24%-79.32% (DeLong)
##   threshold sensitivity specificity
## 1 0.2460815    71.01828    76.19048
## Setting levels: control = FALSE, case = TRUE
## Setting direction: controls > cases

## 
## Call:
## roc.formula(formula = SynQuest_2 ~ GA_lineInter, data = ds, percent = TRUE,     ci = TRUE, boot.n = 100, ci.alpha = 0.9, stratified = FALSE,     plot = TRUE, auc.polygon = TRUE, max.auc.polygon = TRUE,     grid = TRUE, print.auc = TRUE, show.thres = TRUE)
## 
## Data: GA_lineInter in 6960 controls (SynQuest_2 FALSE) > 33669 cases (SynQuest_2 TRUE).
## Area under the curve: 78.41%
## 95% CI: 77.85%-78.96% (DeLong)
##   threshold sensitivity specificity
## 1 0.2460815    70.54264       76.25
## Setting levels: control = FALSE, case = TRUE
## Setting direction: controls > cases

## 
## Call:
## roc.formula(formula = SynQuest_2 ~ GA_lineInter, data = ds, percent = TRUE,     ci = TRUE, boot.n = 100, ci.alpha = 0.9, stratified = FALSE,     plot = TRUE, auc.polygon = TRUE, max.auc.polygon = TRUE,     grid = TRUE, print.auc = TRUE, show.thres = TRUE)
## 
## Data: GA_lineInter in 6177 controls (SynQuest_2 FALSE) > 34452 cases (SynQuest_2 TRUE).
## Area under the curve: 78.58%
## 95% CI: 77.99%-79.17% (DeLong)
##   threshold sensitivity specificity
## 1 0.2460815    69.94949    78.87324
## Setting levels: control = FALSE, case = TRUE
## Setting direction: controls > cases

## 
## Call:
## roc.formula(formula = SynQuest_2 ~ GA_lineInter, data = ds, percent = TRUE,     ci = TRUE, boot.n = 100, ci.alpha = 0.9, stratified = FALSE,     plot = TRUE, auc.polygon = TRUE, max.auc.polygon = TRUE,     grid = TRUE, print.auc = TRUE, show.thres = TRUE)
## 
## Data: GA_lineInter in 5133 controls (SynQuest_2 FALSE) > 35496 cases (SynQuest_2 TRUE).
## Area under the curve: 78.03%
## 95% CI: 77.42%-78.63% (DeLong)
##   threshold sensitivity specificity
## 1 0.2460815    68.87255    81.35593
## Setting levels: control = FALSE, case = TRUE
## Setting direction: controls > cases

## 
## Call:
## roc.formula(formula = SynQuest_2 ~ GA_lineInter, data = ds, percent = TRUE,     ci = TRUE, boot.n = 100, ci.alpha = 0.9, stratified = FALSE,     plot = TRUE, auc.polygon = TRUE, max.auc.polygon = TRUE,     grid = TRUE, print.auc = TRUE, show.thres = TRUE)
## 
## Data: GA_lineInter in 4263 controls (SynQuest_2 FALSE) > 36366 cases (SynQuest_2 TRUE).
## Area under the curve: 77.93%
## 95% CI: 77.27%-78.59% (DeLong)
##   threshold sensitivity specificity
## 1 0.2460815    67.94258    83.67347
## Setting levels: control = FALSE, case = TRUE
## Setting direction: controls > cases

## 
## Call:
## roc.formula(formula = SynQuest_2 ~ GA_lineInter, data = ds, percent = TRUE,     ci = TRUE, boot.n = 100, ci.alpha = 0.9, stratified = FALSE,     plot = TRUE, auc.polygon = TRUE, max.auc.polygon = TRUE,     grid = TRUE, print.auc = TRUE, show.thres = TRUE)
## 
## Data: GA_lineInter in 4002 controls (SynQuest_2 FALSE) > 36627 cases (SynQuest_2 TRUE).
## Area under the curve: 77.43%
## 95% CI: 76.74%-78.12% (DeLong)
##   threshold sensitivity specificity
## 1 0.2677348    68.40855     82.6087
## Setting levels: control = FALSE, case = TRUE
## Setting direction: controls > cases

## 
## Call:
## roc.formula(formula = SynQuest_2 ~ GA_lineInter, data = ds, percent = TRUE,     ci = TRUE, boot.n = 100, ci.alpha = 0.9, stratified = FALSE,     plot = TRUE, auc.polygon = TRUE, max.auc.polygon = TRUE,     grid = TRUE, print.auc = TRUE, show.thres = TRUE)
## 
## Data: GA_lineInter in 3654 controls (SynQuest_2 FALSE) > 36975 cases (SynQuest_2 TRUE).
## Area under the curve: 76.44%
## 95% CI: 75.7%-77.17% (DeLong)
##   threshold sensitivity specificity
## 1 0.2677348    67.76471    80.95238
## Setting levels: control = FALSE, case = TRUE
## Setting direction: controls > cases

## 
## Call:
## roc.formula(formula = SynQuest_2 ~ GA_lineInter, data = ds, percent = TRUE,     ci = TRUE, boot.n = 100, ci.alpha = 0.9, stratified = FALSE,     plot = TRUE, auc.polygon = TRUE, max.auc.polygon = TRUE,     grid = TRUE, print.auc = TRUE, show.thres = TRUE)
## 
## Data: GA_lineInter in 3132 controls (SynQuest_2 FALSE) > 37497 cases (SynQuest_2 TRUE).
## Area under the curve: 74.9%
## 95% CI: 74.09%-75.72% (DeLong)
##   threshold sensitivity specificity
## 1 0.2677348    67.05336    80.55556
## Setting levels: control = FALSE, case = TRUE
## Setting direction: controls > cases

## 
## Call:
## roc.formula(formula = SynQuest_2 ~ GA_lineInter, data = ds, percent = TRUE,     ci = TRUE, boot.n = 100, ci.alpha = 0.9, stratified = FALSE,     plot = TRUE, auc.polygon = TRUE, max.auc.polygon = TRUE,     grid = TRUE, print.auc = TRUE, show.thres = TRUE)
## 
## Data: GA_lineInter in 2697 controls (SynQuest_2 FALSE) > 37932 cases (SynQuest_2 TRUE).
## Area under the curve: 75.28%
## 95% CI: 74.4%-76.17% (DeLong)
##   threshold sensitivity specificity
## 1 0.2677348    66.74312    83.87097
## Setting levels: control = FALSE, case = TRUE
## Setting direction: controls > cases

## 
## Call:
## roc.formula(formula = SynQuest_2 ~ GA_lineInter, data = ds, percent = TRUE,     ci = TRUE, boot.n = 100, ci.alpha = 0.9, stratified = FALSE,     plot = TRUE, auc.polygon = TRUE, max.auc.polygon = TRUE,     grid = TRUE, print.auc = TRUE, show.thres = TRUE)
## 
## Data: GA_lineInter in 2262 controls (SynQuest_2 FALSE) > 38367 cases (SynQuest_2 TRUE).
## Area under the curve: 76.55%
## 95% CI: 75.67%-77.43% (DeLong)
##   threshold sensitivity specificity
## 1 0.2677348    66.21315    84.61538
## Setting levels: control = FALSE, case = TRUE
## Setting direction: controls > cases

## 
## Call:
## roc.formula(formula = SynQuest_2 ~ GA_lineInter, data = ds, percent = TRUE,     ci = TRUE, boot.n = 100, ci.alpha = 0.9, stratified = FALSE,     plot = TRUE, auc.polygon = TRUE, max.auc.polygon = TRUE,     grid = TRUE, print.auc = TRUE, show.thres = TRUE)
## 
## Data: GA_lineInter in 1479 controls (SynQuest_2 FALSE) > 39150 cases (SynQuest_2 TRUE).
## Area under the curve: 75.57%
## 95% CI: 74.44%-76.7% (DeLong)
##   threshold sensitivity specificity
## 1  0.332956    67.11111    82.35294
## Setting levels: control = FALSE, case = TRUE
## Setting direction: controls > cases

## 
## Call:
## roc.formula(formula = SynQuest_2 ~ GA_lineInter, data = ds, percent = TRUE,     ci = TRUE, boot.n = 100, ci.alpha = 0.9, stratified = FALSE,     plot = TRUE, auc.polygon = TRUE, max.auc.polygon = TRUE,     grid = TRUE, print.auc = TRUE, show.thres = TRUE)
## 
## Data: GA_lineInter in 1218 controls (SynQuest_2 FALSE) > 39411 cases (SynQuest_2 TRUE).
## Area under the curve: 80.54%
## 95% CI: 79.55%-81.54% (DeLong)
##   threshold sensitivity specificity
## 1  0.332956    67.10817    92.85714
## Setting levels: control = FALSE, case = TRUE
## Setting direction: controls > cases

## 
## Call:
## roc.formula(formula = SynQuest_2 ~ GA_lineInter, data = ds, percent = TRUE,     ci = TRUE, boot.n = 100, ci.alpha = 0.9, stratified = FALSE,     plot = TRUE, auc.polygon = TRUE, max.auc.polygon = TRUE,     grid = TRUE, print.auc = TRUE, show.thres = TRUE)
## 
## Data: GA_lineInter in 609 controls (SynQuest_2 FALSE) > 40020 cases (SynQuest_2 TRUE).
## Area under the curve: 90.14%
## 95% CI: 89.72%-90.56% (DeLong)
##   threshold sensitivity specificity
## 1  2.581127    84.34783         100
## R version 4.3.3 (2024-02-29)
## Platform: aarch64-apple-darwin20 (64-bit)
## Running under: macOS 15.5
## 
## Matrix products: default
## BLAS:   /Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/lib/libRblas.0.dylib 
## LAPACK: /Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/lib/libRlapack.dylib;  LAPACK version 3.11.0
## 
## locale:
## [1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
## 
## time zone: Europe/Zurich
## tzcode source: internal
## 
## attached base packages:
## [1] stats     graphics  grDevices utils     datasets  methods   base     
## 
## other attached packages:
##  [1] pROC_1.18.5       ggalluvial_0.12.5 ggridges_0.5.6    ggplot2_3.5.2    
##  [5] papaja_0.1.3      tinylabels_0.2.5  dplyr_1.1.4       tidyr_1.3.1      
##  [9] readxl_1.4.5      readr_2.1.5      
## 
## loaded via a namespace (and not attached):
##  [1] gtable_0.3.6       jsonlite_2.0.0     compiler_4.3.3     Rcpp_1.0.14       
##  [5] tidyselect_1.2.1   jquerylib_0.1.4    scales_1.4.0       yaml_2.3.10       
##  [9] fastmap_1.2.0      plyr_1.8.9         R6_2.6.1           labeling_0.4.3    
## [13] generics_0.1.4     knitr_1.50         tibble_3.3.0       RColorBrewer_1.1-3
## [17] bslib_0.9.0        pillar_1.10.2      tzdb_0.5.0         rlang_1.1.6       
## [21] cachem_1.1.0       xfun_0.52          sass_0.4.10        cli_3.6.5         
## [25] withr_3.0.2        magrittr_2.0.3     digest_0.6.37      grid_4.3.3        
## [29] rstudioapi_0.17.1  hms_1.1.3          lifecycle_1.0.4    vctrs_0.6.5       
## [33] evaluate_1.0.3     glue_1.8.0         farver_2.1.2       cellranger_1.1.0  
## [37] rmarkdown_2.29     purrr_1.0.4        tools_4.3.3        pkgconfig_2.0.3   
## [41] htmltools_0.5.8.1